In the digital video arts, analog images, either stored or live, are electronically "captured" by video digitizing equipment into a series of digital numbers, each corresponding to a different one of a plurality of picture elements or "pixels" which comprise the image. Each such number carries information about the color components which, when combined, comprise the color of the particular pixel.
Due to limited system memory, the number of possible such colors is itself limited to a finite number or "palette" of colors. A digital color monitor is normally provided for converting these number into visible pixels which, in the aggregate, form a digitized video image on the screen corresponding to the analog image.
During the aforementioned capture of video images, the video monitor is used to focus, frame, and adjust color. Only after capture and a long palette mapping procedure can the image be seen on the computer monitor using a color look-up table. The long time required to map an analog image into the finite digital palette precludes the use of the computer monitor as the live monitor. Thus, in prior art procedures, a system operator must own and run both a video monitor and a computer monitor, resulting in unnecessary duplication of hardware expenses. More seriously, however, differences in color between the video and computer monitors are not visible while the image is being adjusted, and most video monitors do not show the edges of the image for accurate framing.
A common problem in image display briefly mentioned above is to "map" a full color image to a printer, display, or memory capable of only a limited number of colors. The process of reducing the image to contain only colors in the aforementioned set of limited colors or "palette" is known in the art as mapping to a palette. A typical digital display:might, for example, have 256 palette colors whereby a single digital byte would represent one pixel in the mapped image.
Several methods are known in the art for mapping. In one form known as dithering, a dither pattern is added to the pure image, and each pixel of the summed image is rounded to the nearest color in the palette. Turning to FIG. 4, illustrated therein is an example of dithering for the simple case of a single scan line signal 50 of a monochrome image having only two colors in the palette represented by the portion of the waveform 52 above and below the reference line 53, respectively. Dithering effectively alters the "duty cycle" of each color to create the illusion of intermediate colors. A typical application of this technique is in the printing of magazines which employ dithering to make half tone dots simulating shades of gray differing relative proportions of black dots being utilized. Still referring to FIG. 4, by combining the dithered signal 54 (corresponding to dither 52) and the signal 56 (corresponding to signal 50), the additive combination thereof corresponding to the portion of dither 54 biased upwards by the signal 56 may be seen to be the portion of dither 54 extending above the reference line 55. This resulting mapped signal is thus seen depicted as signal 58 in FIG. 4.
Different dither patterns were known in the prior art. In two dimensions, such a dither pattern usually consisted of a dither matrix which repeated through the image. Turning to FIG. 5 now, if this dither matrix 60 or 62 has four elements, the dither can add two bits to the effective gray scale, as shown therein. Thus, as an example, an additive modulation of 0.01 in matrix 60 (upper left corner) with modulation 0.10 (upper left corner) in matrix 62 results in a net modulation of 0.11 (upper left corner) in matrix 64 which may be seen to be one of the four element matrixes of the overall image matrix 65 after having been modulated by matrices 60 and 62. It can be shown that if the dither matrix such as matrix 60 has 16 elements, it can add 4 bits. These bits are to the right of the binary point, wherein unity is the step between discrete colors in the palette.
The least noticeable pattern for dithering typically is a checkerboard because the sampling noise is at the diagonal extremes of the spatial frequency bandwidth. The most significant bit of dither is therefore assigned to the checkerboard pattern. For cathode ray tube or "CRT" displays, the next least noticeable pattern has been found to be a vertical stripe so the next bit is assigned to vertical stripes as portrayed in FIG. 5.
The problem with prior art dither techniques is that the dither pattern itself distracted from the image. 0n a digital image monitor, which normally has limited resolution, the dither pattern provided a major distraction preventing the use of the image for focusing or aligning fine detail.
Prior methods of palette mapping have been provided in the art to improve image color utilizing various well known techniques such as error diffusion. Although they may provide desired improvements in full color imaging, the mapping techniques were so computationally complex as to render them of little practical use in many applications wherein time was not available after the initial image capture for the processing and subsequent viewing of the image to determine if it was captured correctly in the first place. In other words it was not possible to adjust the image and compensate for differences in color in computer monitors in real-time whereby the computer monitor itself could actually be utilized to focus, frame, and adjust color as well as for final display and use of the image.
Accordingly, in the prior art, with a digital image display using mapped images, it was not possible to achieve both the required quality and the required speed at the same time to use the display as a real-time monitor. The subject invention solves this problem by providing a dither system in which the technique employed reduces the aforementioned visual distraction.
Turning now to FIG. 6, similar to FIG. 4, there is shown a magenta and green single scan line, respectively. In accordance with techniques of the present invention for reasons hereinafter described, dithering is provided for these green and magenta colors shown as waveforms 70 and 72. In like manner to FIG. 4, the combination of the signals 66, 68, and the corresponding dithers 70, 72 shown as signal 78 and dithers 74, 76, resulted in corresponding green and magenta mapped signals 80, 82, respectively. In a color image, prior art however only applied this dither to the red, green, and blue planes. In contrast, the invention applies the same pattern matrix to red and blue, and the complement to green for reasons that will be hereinafter described in detail.
Referring now to FIG. 8 in contrast to FIG. 7, in accordance with the invention the green and magenta signals 92, 94, respectively, would switch out of phase with each other, thereby cancelling the dither frequency from the luminance 96, thereby removing the dither pattern distraction from luminance. The chrominance or color, on the other hand now switches between green and magenta as shown by chrominance signal 98. Thus it may be seen that the complementary dither trades luminance noise for chrominance noise.
Turning now to FIG. 7, in the prior art, if the original image was a gray scale, then the green and magenta colors would turn on and off together in phase, represented by waveforms 84 and 86, respectively. As a result, there would be no deviation in chrominance or color at all, as shown by waveform 90. However, the luminance would switch at the dither frequency, shown as luminance waveform 88, thereby producing the familiar half tone dot pattern common to dither techniques.
Referring now to FIG. 9, this tradeoff of FIG. 8 is portrayed relative to the human visual system. Depicted at the top of FIG. 9 is a plot of human eye sensitivity curves. In these plots, a frequency of 10 cycles per degree corresponds to 20 pixels per degree or 20 pixels per 4.43 millimeters viewed at 10 inches, resulting in 640.times.480 pixels on an IBM model 8515 monitor viewed from 17 inches. Accordingly, the scale and magnitudes shown in FIG. 9 are approximately quantitative and representative of real images.
Still referring to the topmost graph, it will be noted that the resolution of the human eye differs for "color" or chrominance relative to "black and white" or luminance, as shown by the sensitivity graphs 100 for luminance and 102 for chrominance. Effectively one may conclude from the plots that the eye is relatively insensitive to color details.
Referring now to the middle plot of FIG. 9, relative to the luminance and chrominance waveforms 88, 90, respectively, of FIG. 7 in the prior art, it will thus be seen that the luminance noise 106 of the prior art increases substantially with frequency tracking the luminance waveform 88, whereas the chrominance noise 104 increases less substantially, consistent with the chrominance waveform 90 of FIG. 7. Comparing the upper and middle graphs of FIG. 9, it will thereby be apparent that as a result of prior art dithering techniques, because the eye physiologically sees high frequency luminance noise much more effectively than chrominance noise, the perceived noise is greater by employing the prior art techniques. In other words, the sensitivity of the human eye to luminance noise increases with frequency as does the noise resulting from prior art techniques thereby exacerbating the problem. It would thus appear, as will be hereinafter detailed in the description of the preferred embodiment, that trading the high frequency luminance noise for chrominance noise in accordance with the teachings of the invention has been found to provide startlingly favorable results. Because the eye simply does see the high frequency chrominance noise as well, accordingly the perceived noise is greatly reduced.
There is yet another problem associated with prior art approaches briefly mentioned heretofore. It is conventional during the capture of video images for editing and composition purposes prior to arriving at a final image to provide a video monitor for use in real time during the capture process to focus, frame, and adjust color. Due to the long time required to effect palette mapping and associated dithering procedures through color lookup tables and the like just described, this effectively precluded use of the computer monitor itself as the "live monitor" during the capture process for adjusting focusing frame, adjust color, etc.
Accordingly, due to this limitation in prior art procedure precluding use of a digital computer monitor during the capture process, this in turn required that the individual performing the capturing and image adjustment must have access simultaneously to both the video monitor for such capture/editing process and the computer monitor as well.
In addition to the obvious expense of having two monitors associated with prior techniques, even more seriously differences in color representation between the two separate video and computer monitors obviously could not be visible while the image was being adjusted since the adjustment would occur in real time while the video monitor was being employed and the final display would occur thereafter on the computer monitor after mapping from the video monitor display. Additionally, most video monitors do not directly show edges of an image as desired for accurate framing as they would desirable appear on the computer monitor and final image, thereby causing additional problems.
From the foregoing, it will be appreciated that a system and method was needed for palette mapping for digital imagery which would permit display of full color images on hardware with limited palettes. Moreover, improvements in prior art techniques were needed which provided for less perceived noise in the captured image. Still further, it was an object of the invention to provide for improved performance in palette mapping, and to more particularly provide for real time digital imaging with improved palette mapping performance, thereby permitting the Computer monitor to be utilized in real time in the capture and edit process to focus, frame, and adjust color,
These and other objects have been fully met by the subject invention, a description of which hereinafter follows which may be more easily understood with reference to the following drawings wherein: